COMMENT ⊗   VALID 00008 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	.DEVICE XGP
C00003 00003	.SELECT B
C00008 00004	.BEGIN FILL ADJUST
C00013 00005	.SELECT A
C00018 00006	.SELECT 5
C00022 00007	.SELECT 5
C00026 00008	.GROUP SKIP 2
C00030 ENDMK
C⊗;
.DEVICE XGP
.FONT 3 "FIX25"
.FONT 2 "SUP"
.FONT 1 "NGR30"
.FONT 4 "GRK30[MA,JAM]"
.FONT 5 "BASI30"
.FONT 6 "GRKL40[MA,JAM]"
.FONT 7 "SUBSUB"
.FONT 8 "SUB"
.FONT 9 "FIX30"
.FONT A "BASB30"
.FONT B "CLAR30"
.FONT C "BDJ25"
.!XGPCOMMANDS←"/XLINE=10";
.TURN ON "%↑↓";
.EVERY HEADING(,{PAGE},)
.PAGE FRAME 39 HIGH 80 WIDE
.TITLE AREA HEADING LINES 1 TO 2
.AREA TEXT LINES 3 TO 38
.TITLE AREA FOOTING LINE 39
.SELECT 1
.NOJUST
.NOFILL
.NEXT PAGE

.SELECT B
        SIMULATION OF MUSIC INSTRUMENT TONES
.SELECT 1

.BEGIN FILL ADJUST

In this part  of the proposal we  will discuss our approaches  to the
computer simulation of  music instrument tones.  The main goal of our
research is the development of a  powerful, general purpose technique
for the simulation of auditory  signals that will have the perceptual
complexity  and naturalness of the musical  sounds which occur in the
real world.  The fundamental concern  here is  with the synthesis  of
natural  timbres of  the extremely  varied and  highly  complex tones
which occur in music.   Timbre can  be psychoacoustically defined  as
that multidimensional attribute of sound by which  a listener is able
to  differentiate  two auditory  signals  which are  equal  in pitch,
loudness, and duration.  Although a general model does  not presently
exist  for the  perception of  timbre, the  psychoacoustical research
which  has  been done  in  the last  century implicates  a  number of
physical properties  of tone  which influence  timbre, including  the
spectral distribution of energy  and the temporal pattern of spectral
evolution.

Supporting  this  goal  to  develop  a  technique  for  the  computer
simulation  of  natural  tones is  a  concurrent  research effort  to
formulate a  perceptual model  which is  able to  desribe  the  human
processing of musical sounds. The theoretical problem is to establish
a  set of acoustical dimensions, those  which are actually salient in
the perception  of musical  timbre,   and to  design a  computational
algorithm that enables the user to exert the highest level of control
with respect to these dimensions for the purposes of simulation.  The
empirical problem which follows is the determination of those aspects
of  the signal  which are  actually important  in the  perception and
identification of a  sound. Necessarily  included is a  study of  the
distinctive  features  of signals,    the  investigation of  physical
conditions  which contribute to the naturalness  of a signal. Related
research  should  examine  the  general  characteristics   of  timbre
perception,  looking  into  the  effects  of such  phenomena  as  the
categorical identification of musical sounds.

The discussion which  follows is concerned  with a desription  of our
systematic approach to a general model for simulation, which is based
on  perceptual  validation  at  every  step.  Two  initial strategies  
for  simulation,  the  analysis-based  additive  synthesis  and   the
frequency-modulation  synthesis, are seen to  converge on this model.
The first method is concerned with the procedures  of data reduction,
in that we start with  the most complete, complex information about a
real signal through its analysis.  We then systematically step in the
direction of the most  simple representation of the signal  which can
be  used successfully to  synthesize a perceptual  replication of the
original tone.   To this  end we examine  the perceptually  important
aspects of physical signals. 
.END
.BEGIN FILL ADJUST
The second  approach towards  a model  for simulation  begins with  a
simpler,    more  easily-controlled  process,    frequency-modulation
synthesis of sound,   which  allows the user  to directly  manipulate
aspects of the signal that we have found to be very powerful in terms
of certain perceptual cues for music instrument tones. The success of
this method first  came as a surprise  to many, in that  the physical
waveform that  it generates is strikingly different  from any natural
signals.  However,  upon inspection, the reasons for this success may
be determined,  and we may thereby learn what physical dimensions are
important  in  the perceptual  achievement.   The  direction  of this
approach is to increase the  complexity of the process,  until  there
is control  over a very wide  set of features which  occur in natural
tones.

The approaches described are  not found to proceed independent of one
another. Findings in one technique can immediately be applied to  the
other, and a system of cross-verification is thereby established. The
ultimate  model  for simulation  will  draw from  findings  from both
approaches. A common  aspect of  both approaches is  the concern  for
perceptual   verification  of   the  particular   results  at   hand.
Experimental  methods of  perceptual psychology are  employed for the
rigorous verification of the success of simulation,  in  terms of the
discriminability of a simulation from  real tones and in terms of the
naturalness of simulation. 

The ultimate  aim  of our  research is  the  development of powerful,
general purpose  algorithms  for the  computer simulation  of natural
tones. Accordingly, we will investigate the general properties of the
perceptual processing  of such  signals.  The  general model  for the
perception  of different sets  of tones will  provide information for
the  construction   of  perceptually-based  higher-order   simulation
algorithms.   We employ a spatial model  for the subjective structure
of  the  perceptual  relationships  between  signals.    Research  is
directed at  uncovering the dimensionality  of the  subjective space,
the psychophysical relationships which are structurally correlated to
this space,  and the properties of the space.  The existence  of such
constraints  as categorical  boundaries  will be  investigated in  an
attempt to  assess the continuity of the subjective space for timbre.
In the  same regard,  we will  also  examine the  effects of  musical
training or context on the structure of the space.  The model will be
evaluated by our ability  to predict the mappings  of real and  novel
tones.
.END
.GROUP SKIP 3
.SELECT A
                    ANALYSIS-BASED ADDITIVE SYNTHESIS
.SELECT 1

.BEGIN FILL ADJUST
This section  will be devoted  to a discussion  of the  simulation of
musical  tones by way  of analysis-based additive  synthesis. We will
first  indicate  the  computational  methods  involved  in   additive
synthesis, and then go on to  describe the analysis of of real tones,
which  provides  the  initial information  for  simulation.   Graphic
techniques which aid the researcher in the examination of the complex
physical properties of tones will be described.

Next  we  will  discuss   the  perceptual  validation  of  the  basic
analysis-synthesis model.    We  will  then go  on  to  describe  the
direction of our current research, the reduction  of the very complex
physical data which is obtained from the analysis of real tones. Data
reduction will be found to depend on an empirical verification of the
perceptual fitness  of measures  taken: the success  or failure  of a
current  data reduction strategy  will contribute to  a general model
for the perceptual processing of musical tones;  the model,  in turn,
will direct  the next stage of data  reduction. We will also indicate
future research which  is planned  for the approach  to our  ultimate
end:  a powerful,    general,   and  easily-controlled algorithm  for
simulation  which is  based on  a comprehensive perceptual  model for
natural tones.
.END
.NEXT PAGE
.SELECT 3
SYNTHESIS AND ANALYSIS TECHNIQUES
.GROUP SKIP 1
.SELECT 5
synthesis
.SELECT 1
.BEGIN FILL ADJUST
In  additive  synthesis  we  physically  model  the  digitized  sound
waveform  as a sum  of sinusoids with  slowly time-varying amplitudes
and phases.  Equation (1) summarizes this formulation.  The sinusoids
in equation (1) represents the partial tones.
.END


          M
(1)  F%8α%1 = %6S%1 A%8j%1 sin(%4w%8j%1hα+%4q%8j%1)
         j=1

Notation:    F%8α%1 is the sampled, digitized waveform at time αh
	     h is the time between consecutive samples
	     A%8j%1 is the amplitude of the jth partial tone
	         and is assumed to be slowly varying with time
	     %4q%8j%1 is the phase of the jth partial tone
	         and is assumed to be slowly varying with time

.BEGIN FILL ADJUST
One can see that from this model, if we can determine the
parameters A%8j%1 and %4q%8j%1 of a tone from a musical instrument,
we can easily synthesize the waveform F%8α%1 from those parameters
by use of equation (1). The degree to which this form of synthesis
has been successful will be discussed below.
.END
.GROUP SKIP 2
.SELECT 5
analysis for additive synthesis
.SELECT 1
.BEGIN FILL ADJUST
We now turn to the problem of determining the parameters
A%8j%1 and %4q%8j%1 of a music   instrument tone. To aid the
analysis, we must assume the frequencies of the partial tones,
%4w%8j%1, are nearly harmonically related. That is, there is some
frequency, %4w%1, such that %4w%8j%1 is approximately j%4w%1. We shall
call this frequency %4w%1 the fundamental frequency of the tone.

The method we have found most useful we call the "hetrodyne
filter." This is described in detail in reference [**Moorer**] and is
derived briefly in appendix A. Basically, the method is as follows:
First, compute the following two summations at each point in time α.
.END

         α+N-1
(2)  %9a%8j%8α%1 = %6S%1 F%8i%1sin(j%4w%80%1ih+%4f%80%1)
          i=α

         α+N-1
(3)  %9b%8j%8α%1 = %6S%1 F%8i%1cos(j%4w%80%1ih+%4f%80%1)
          i=α

.NEXT PAGE
From these, we calculate two more sequences:

(4)     A%8jα%1 = (%9a%8jα%22%1+%9b%8jα%22%1)%21/2%1

(5)     %4f%8jα%1 = atan(%9a%8jα%1/%9b%8jα%1)

.BEGIN FILL ADJUST
The summations are taken to be over one period of a sinusoid
of frequency %4w%80%1, that is, Nh%4w%80%1 = 2π. This places
somewhat of a restriction on the frequency of analysis, %4w%80%1,
because in the discrete domain, the period, N, is restricted
to integral values. This has not proved a problem in our experience.

If the partial tones are nearly harmonically related, if the
parameters of the tone vary slowly with time, and if %4w%80%1 is nearly
equal to the fundamental frequency of the tone, then A%8jα%1 and
%4f%8jα%1 will indeed be approximations to the actual amplitudes and
phases of the partials of the tone under analysis.

Let us review the procedure. First, we do the computations indicated in
equations (2), (3), (4), and (5) for each of the partials of a
tone, over the entire time interval spanned by the tone. The output
A%8jα%1 and %4f%8jα%1 may then be used in equation (1) to synthesize a new
tone that hopefully retains much of the character of the original.
.END
.GROUP SKIP 2
.SELECT 5
graphic techniques
.SELECT 1
.BEGIN FILL ADJUST
As aids to the researcher, we have designed several different methods
for displaying  the results of analysis. The  output of the hetrodyne
filter can, of course, be displayed as a number of isolated amplitude
and phase plots,  covering the individual components, as  is shown if
figure 1  for the first sixteen harmonics of a violin tone. The total
duration of the tone  is about 400 milliseconds and  it's fundamental
frequency  is about 311  Hz. It  has been  found most  informative to
simultaneously view the entire set of harmonics together.  One method
is the use  of perspective plots. Figure 2  shows such a plot  of the
amplitudes of the  partials of the same violin tone.  The fundamental
harmonic is in the background of  the picture, the highest is in  the
foreground.  This allows  us to  more readily  discover relationships
among  the harmonics.   This  perspective plot  can be
spatially rotated on-line,  so  that the observer is able to  see the
three-dimensional representation from any  angle.  This has been very
helpful in getting a more comprehensive understanding of the behavior
of the partials of a tone as a function of time.

Another form of showing  the temporal evolution of the  partials of a
tone  is the  sequential line-spectrum plot.   This  strictly on-line
display presents two-dimensional frequency by amplitude plots  of the
partials  for a  given  instant in  time.   The  observer traces  the
changes   in  these  parameters  through   a  sequential  display  of
instantaneous values from the beginning to the end of the tone.  This
animation  technique has  been  found to  reveal  further information
gathered from analysis.

.NEXT PAGE
A fourth way of examining the output of the hetrodyne filter, inspired
by the  sonogram,  is given in figure 3.  Here, the thickness of each
bar is proportional to the log of the amplitude of that harmonic. The
vertical   position  represents   its  instantaneous   frequency,  as
determined from the phase drift of the harmonic.  The utility of this
display is it's representation of the phase  information with respect
to amplitudes.
.END
.GROUP SKIP 2
.SELECT 3
PERCEPTUAL VALIDATION OF ANALYSIS-SYNTHESIS STRATEGY
.SELECT 1
.BEGIN FILL ADJUST
It is  necessary  to confirm  the utility  of our  analysis-synthesis
strategy  for the  simulation of  musical tones on  the basis  of the
perceptual success of the resultant auditory simulation.  That  is to
say,   we must  establish that  the signal which  is produced  by our
analysis-synthesis technique is indiscriminable from the recording of
the original sound.  The crutial test,  then,  is a comparison of the
sound which has been produced by additive synthesis with the original
musical tone that  was analyzed with  the hetrodyne filter.  Informal
experimentation has shown that the analysis-synthesis method produces
an extremely  convincing replication of the  original signal, and the
perceptual  validity  of  the  method  has  been  thereby  informally
established.  This  is  in  agreement  with  the  findings  of  other
investigators   who  have   attempted  to  verify   similar  analysis
techniques by  comparing  tones synthesized  from analysis  with  the
original signals (Risset, 1966; Freedman, 1967, 1968; Beauchamp, 1969).

Among  the  types   of  natural  musical  signals  which   have  been
sucessfully  simulated by the analysis-synthesis  procedure are tones
from the string,   woodwind,   and brass  families of the  orchestra.
Specifically, we have been able to reproduce tones of various pitches
and durations from the following instruments:
.END
.GROUP SKIP 1
.NOFILL
.NOJUST
	violin,  viola,  cello,  string bass,  trumpet,  trombone,
        french horn,  baritone horn,  oboe,  english horn,  flute,
	Bb clarinet,  alto clarinet,  bass clarinet,   alto flute,
	alto sax,  soprano sax,  bassoon.
.GROUP SKIP 1
.BEGIN FILL ADJUST
The  comparisons  between  the  original  recorded  tones  and  their
respective  simulations  by  experienced  listeners,   composers  and
acousticians, have demonstrated the perceptual validity and potential
power of our analysis-synthesis technique.
.END
.GROUP SKIP 2